A Journey Through Musical Media and Spotify

Introduction

Media stimulates all kinds of senses. In entertainment media this is often done through screens which entertain our visual senses. Our auditory senses are just as important. This is where music comes in the picture. It not only gives assistance to the overal atmosphere, but can also contradict it. This can give us an experience that can’t be achieved with only (moving) pictures. Music in video games can contain many different music genres and sounds. They can be orchestral, jazz, electronic and more. In my research I want to explore this musical ‘genre’. When playing games and listening to the music I could hear differences in the types of music that were played in games. Based on that I tried to make two distinct group of video games, of which I could compare the music of. These groups that I chose are triple A and Indie games. These two groups are often used to distinct types of video games in the video game community. Indie games are characterised by a small group of developers without the financial support of a publisher. Triple A games can be seen as the opposite of that, in which that they are characterised by a game developed and published by a mid-sized or major publisher. To put it in the context of film, triple-A are analogous to blockbuster movies and indie games can be seen as independent movies.

Because of this difference in financial budget, one can imagine that there would be a difference in what kind of music are produced and put in the game. Among other things, one can hear that usually in triple A games, the music consists of orchestral pieces and other high-budget components like choirs. Indie games on the other hand usually make use of sounds that are easily created by a computer like chiptune and midi. Furthermore, in recent developments, triple A games have aimed to have a higher sense of immersion. They tried to achieve this by simulating real life as much as they could, having life-like scenery and humanistic character models. This pursuit of realism has also been translated in its music. In real life, there is no soundtrack to accompany your day to day activity. Just like in films, triple A game developers have chosen for music that you do not really notice, but do add to the overall experience. Because of this, I also wanted to compare video game music to high-budget/blockbuster films. I could have also chosen to add an additional independent film soundtrack, but as these are not as popular on Spotify I chose to not add this.

The Corpus

So, my corpus consists of different types of video game and movie soundtracks. The way I choose these game soundtracks was done with some thinking behind it. Often the distinction between Triple A and Indie games can be vague: when is a game big enough to be triple A? What kind of video game company is ‘independent’? So when choosing the games to be in my corpus I made sure that the games belonged to companies that were notorious for being triple A or indie. That meant that I choose games from big studio’s like Blizzard, EA, Activision and more. Lastly I also picked out some film soundtracks to compare these game soundtracks to. It has only been a couple of years that it is common practice to put film soundtracks on Spotify. These are mostly also soundtracks from big, popular Hollywood movies. I made a selection from these recent film soundtracks and compared it to the two types of game soundtracks.

Differences and Similarities: Classification

This is some text

Graph Analysis

In this graph we can see the most important variables that the random forest uses to differentiate between the different media forms. What is remarkable is that all the timbre coefficients are present and high up in the list. Namely, coefficient 4 is the most important variable. Spotify refers to this coefficient as “sounds with a stronger attack”. If this is put into context this makes sense. The timbre differences between the different soundtracks are big. As I mentioned at the start triple A soundtracks tend to have more of an orchestral sound, while indie make use of computer-generated sounds. This is of course not true for every game, but in this case it can apply well when looking at the graph.

Other variables other than timbre that are present in the graph are energy and valence. This is also interesting as it implies a difference in emotion and mood in the soundtracks. In practice, I think that indie soundtracks commonly have a wide range of emotion and energy in their music. This means it both has cheerful, happy music, but also more dramatic, serious music. With the triple A and movie soundtracks I feel that this range is smaller, tending to be overall more suspenseful, dramatic and minor. This is of course all speculations. In the next tabs I will be exploring timbre, energy and valence more by making visualizations and see what visible difference there actually are.

Confusion Matrix Analysis

In this confusion matrix you can see how R sorts and predicts the different tracks. This is done by looking at the top 5 elements in the random forest graph. This is C04, energy, C09, C06 and C01. Here we can see that R does an okay job differentiating the tracks and putting them in the right playlists. Around 50% of the time this is done correctly. When this is not done correctly, often there is a tendency to group triple A tracks to film tracks and vice versa. This is almost done 30% of the time. This can confirm my prediction that film and triple A soundtracks are more alike than when compared to indie soundtracks. When looking at the indie group, one can see that when a track is not put in the right group, it often assigns it to the triple A soundtrack. This can make sense, as they both are soundtracks for video games.

Visualising the Audible Media

In this graph I plotted valence against energy. You can see clear patterns emerging from the different genres. Noticeably when looking at the indie soundtrack, a wide variety of points can be seen with both high and low energy and high and low valence, with a small saturation in the low energy and low valence corner in the bottom left. This saturation is more prevalent in de triple A and film soundtracks. These genres do not have that much variety in their energy and valence levels. Especially in the triple A genre one can notice that the top half of the graph is rather unpopulated when compared to the bottom half. This can mean that my predictions made in the classification tab were in the right direction. Indie in this case has music with a wide range in emotions and energies, while film and triple A has a small range.

Besides this it is also interesting to note that even though indie has a wide range in valence and energy, there is still a high amount of tracks with low energy and valence. This can mean that in music for games and movies slower and negative emotion are more conventional than the other way around. A conclusion that one can make up from this is that games and films usually contain dramatic, suspenseful and tense moments, that occur more often than happy, chipper instances. This can be the reason why low valence and energy soundtracks are more prevalent. But these are just quick conclusions that may or may not represent reality. In the next tabs we will compare the soundtracks in different ways that can tell us more about how they differ.

Colourful Pitches: Chromagrams

This is some text

The first chromagram is from the track “Roger and Me” from the triple A game Diablo II. Here one can see that the bright colored bands are at C, Eb and G. This is an indication that the key of this song is C minor. Around two thirds of the graph the Eb colored band switches to E. This can indicate a modulation from C minor to C major.

The second chromagram is from the track “A Messenger Is Needed (introduction)” from the indie game The Messenger. In this chromagram there are also prominent bands that can indicate a key. These bands are E, B and D. This means that the key is either E minor or E major. This is not clear because the major or minor third is not known. This can either be G or G sharp

Lastly, the last chromagram is from the track “Old Tech” from the movie Avengers: Infinity War. Interestingly the two most prominent bands are C and C sharp. When listening to the song this makes sense. The song is very dramatic, eerie and dissonant. This is reflected in this chromagram because the most used tones have a interval of a minor second which is usually very dissonant. The track ends more major, with a slow fade in of high strings. It plays a short motive which is an allusion to the Avengers theme. This can explain the high magnitude at G towards the end of the graph.

The chromagram are an interesting representation of the tracks. More melodic tracks like “Roger and me” and “A Messenger is Needed” contain chromagrams where you can look for a key. Tracks that are more abstract and atmospheric are not easily defined by a key and this is also seen in the chromagram.

Colourful Tones: Cepstrograms

This is some text

The first cepstrogram is from the track “Roger and Me” from the triple A game Diablo II. This game makes use of guitars, flutes and electronic sounds. It sounds very mysterious and suspenseful. This mood is created by the guitars which play quick arpeggio’s that blend into one another, together with the electronic sounds. The flutes play the melody after a few introduction bars. This creates a nice contrast as the high and bright flute shines over the guitar accompaniment. One of the coefficients that stand out in the cepstrogram is coefficient 3. Spotify describes this as “the flatness of a sound”. This can be heard in the guitar accompaniment, which do not necessarily have any high or low frequency, but sound kind of muddy when blended together. Other coefficient also stand out in this cepstrogram, but I find it hard to analyze this, as Spotify only describes what coefficients 1 to 4 are.

The second cepstrogram is from the track “A Messenger Is Needed (introduction)” from the indie game The Messenger. This track is fully composed with chiptune. This is a synthesized retro-computer sound. The track consists of many quick arpeggiated notes with melodies that can be heard from time to time. In this cepstrogram you can see that coefficient 2 is very present in this track. Spotify describes this as “brightness”. This is can be heard in the song because it contains chiptune. Chiptunes are very bright as it contains many overtones. Much of the chiptunes are also high in frequency which can add to the brightness.

Lastly, the last chromagram is from the track “Old Tech” from the movie Avengers: Infinity War. This track is very dark and ominous played by string, low brass and bombastic percussion. It ends with a more hopeful tone as the strings play a small motive alluding to the Avengers. The cepstrogram for this track is not very clear. It more or less the same color for every coefficient. Only coefficient 2 is darker. This is heard in the music as the track is not bright at all. There are many low string and brass instruments playing which make it sound more muddy.

Colourful Keys: Keygrams

I found it harder to analyse these keygrams, because I had found it did not represent the tracks accurately. It was also not precise, because many times all the different keys are present on the same level, even though there clearly only one key was audible in the song. There are also instances in every keygram where for a short period of time all the keys would be active. I could not really make this up from the tracks as well. The exception for this can be the second keygram which is from the track “A Messenger Is Needed (introduction)” from the indie game The Messenger. In the beginning of this track, many quick arpeggio’s can be heard that blend into one another. I suppose you can make up many different keys from this specific segments.

Colourful Groupings: Clustering

This is a dendrogram of my 3 types of soundtrack. It would be too big if a dendrogram was made with my original playlist, so I cut it down to 50 songs per soundtrack. I manually coloured every branch to the corresponding genre. At first glance there seems to not be any clear division between the tracks. They seem to not be very distinctly grouped. There are some small clusterings of five songs of the same type of soundtracks sprinkled throughout the dendrogram. This can mean that R can see some differences and similarities between the soundtracks. When looking at the larger clusters there does not seem to be any existing pattern. Perhaps this can indicate that there are not very big differences between the soundtrack

Conclusion

After all these visualizations and analyses what can we finally say about game and film music? First of all let’s summarize what we learned from our data. We began with the classification that guided us into the direction of looking at timbre, energy and valence. From the confusion matrix we could also see how R predicts the grouping of each individual track. We learned that R did an average job at putting the tracks into the right categories, but when it made mistakes, it tended to put triple A titles with film soundtracks and vice versa. Next there was a visualization of energy plotted against valence. There was a big difference to be noted: indie soundtracks tended to be a lot more diverse than the triple A and movie soundtracks. The graphs for triple A and film however were a lot more visually similar. Lastly we looked at cepstrograms and analysed three different tracks. These cepstrograms showed us how diverse these songs were in their timbre. They were not very similar and were interesting to look at in their own way. Because these are only three tracks, they are not representative for the whole corpus, but can be an insight on how other similar tracks may look like.

Overall, it can be said that film and triple A soundtracks are very similar. This can be observed through their energy and valence levels but also through the confusion matrix, where they were put into categories by observing their timbre coefficient levels and their energy levels. But these different properties are not the only factors when comparing these soundtracks. Because these tracks are used in entertainment media, it also means that there is a certain context tied to the music. This is also an important aspect to note when looking at these soundtracks. Furthermore this is a very small research project. With more time and careful observation and computing, one can be able to make better, accurate and representative results.

This is some text